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This paper provides an overview of the cognitive 
process analysis of tasks used to measure aptitude and intelligence. 
As an illustration of this approach, performance on inductive 
reasoning tasks such as series extrapolation and analogy problems is 
considered In terms of the factors that contribute to item difficulty 
and individual differences in measured aptitude^ The analysis * 
emphasises the development of information processing models that 
specify content ajid process factors that determine the speed and 
accuracy of task performance. Theoretical and empirical efforts lead 
to the postulation of rule complexity and representational 
variability as two general sources of difficulty in inductive 
reasonina tasks. These two factors provide a basis for the analysis 
of individual differences in aptitude test performance and also for 
instrtrctional experimentation designed to influenjce reasoning skill. 
In aeneral, the paper illustrates the implication^ of applying 
theories of cognition to the field of psychometrics • (Author)* 
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Abstract 



This paper provides an overview of the cognitive process analysis of 
tasks used to measure aptitude and intelligence. As an illustration of 
this approach^ performance on inductive reasoning tasks such as series 
extrapolation and analogy problems is considered in terms of the factors 
that contribute to item difficulty and individual differences in measured 
aptitude. The analysis emphasizes the development of information pro- 
cessing models that specify content and process factors that determine 
the speed and accuracy of task performance. Theoretical and empirical 
efforts lead to the pu>stulation of rule complexity and representational 
va riability as two general sources of difficulty in inductive reasoning^ 
tasks. These two factors provide a basis for the analysis of individual 
differences in aptitude test performance and also for instructional ex- 
perimentation designed to influence reasoning skill. In general, the pa- 
per illustrates th^ implications of applying theories of cognition to the 
field of psychometrics. 
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COGNITIVE n^OCESS ANALYSIS OF APTITUDE: 
THE NATURE OF INDUCTIVE REASONING TASKS 



Robert Glaser and Jamos W. Pellegrino 

Learning Rotearch and Development Center 
University of Pittsburgh 

At the present time, cognitive psychologists are studying individ- 
ual differences in intelligence and aptitude in terms of the structures and 
processes hypothesized in the study of cognition and cognitive develop- 
ment. The research carried out along these linos is predicated on the 
assumption that aptitude tests should be viewed as more than primarily 
predictors of achievement. Rather, such tests should assist in identi- 
fying the processes involved in intellectual competence, and further indi- 
cate how these processes can be influenced and utilized to benefit learn- 
ing. In order to contribute to this purpose, we have undei taken a research 
program in which we are attempting to identify directly thf. cognitive pro- 
cessing components of performance on tasks used to assess aptitude. Per- 
formance cn psychometric test tasks that predict success in education and 
training becomes the object of theoretical and empirical analyses. The 
immediate goal is to analyze test tasks, develop process models of task 
performance, and utilize these models as a basis for individual difference 
analysis. The distant goal Is that, based on this knowledge, conditions 
designed for learning could be adjusted to these individual characteristics, 
or instruction could be designed so that it directly or indirectly teaches 
the processes that facilitate learaing. The instructional objective is not 
to train individuals to score higher on tests of mental ability, but to di- 
rectly or indirectly influence the cognitive processes that underlie suc- 
cessful performance both on tests of aptitude and in instructional settings. 
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Ai A gon^ral background for tha lummary report of our rciieiirch* 
wo will briofly mention four operating conitralnts thnt are Important in a 
task-analytic effort aimed at underitanding individual difference! in apti- 
tude procotiei. 

A Framework for Taik Analyaii 

Firfltt lince there ii an extremely large connteUation of piycho<- 
metric taika« one baiic principle for taik lelection ii pervaiivenesi and 
robuatnttia. By thii» we mean identifying a core ict of tasks that fre- 
quently occur in widely used tests* and that have demonstrated consis- 
tent relationships in factor-analytic studies of certain basic aptitude 
constructs* A particular task or set of tasks chosen for analysis should 
have a strong history of reliable association with an aptitude construct 
that is of reasonable generality, and also has consistent predictive validi- 
ty with respect to a criterion performance of signiflcsnt Interest. 

Secon ip in the analysis of any particular aptitude construct, it is 
important to simultaneously consider several tasks or task forms that 
load on that factor. Analysis should consider the various tasks that in- 
tercorrelate and thereby define more completely a substantial set of per- 
formances that comprise an aptitude construct. The analysis of multiple 
tasks should enable us to differentiate general and specific cognitive pro- 
cesses* and thereby help direct us to a level of analysis where research 
can identify the extent of process trainability and transfer effects. 

Third, the analysis of a particular task must also be explicitly 
concerned with explicating the sources of item difficulty that provide the 
baais for individual variation in test performance. Test tasks are com- 
posed of item sets where the individual items vary considerably in diffi- , 
culty as a function of ability or developmental level, and an understanding 
of individual differences in task performance must include a process 
theory of item difficulty. For this purpose, the processes specified as 
tile components of performance must involve a level of analysis that is 
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■ufflciont tQ expUin individual Item chttrActcrlHtlcii, mdlvlduttl uulijtict 
porformAnco, And tho Intorftctlon o£ tiio two. 

A fourth coristfttlnt arlioii from the prlmiry goal» (»f a partlculttr 
roiO*rch program. In our own work, the \iou\ in to develop a theory of 
individual dlfforencoi In aptitude procoaaioii that m^K^t Identify inatruc- 
tlonally tractable component* of cognitive performance. Thlw constraint 
ii a nontrlylal criterion, since It <h not unusual for pnychologUta to «n- 
gago in detailed task analynea ■pecltylniij molei:ular le^'olii of procomilng 
that may have little relevance to Inatructlonai Inaueo. With tills in mind, 
we propoae that the empirical and theoretical results of any particular 
analysis of the cognitive components of a task be evaluated by asking 
whether such results suggest testiible Instructional hypotheses. Such In- 
ternal tests of the task^analytlc effort can be sobering Indications that wo 
h».ve yet to achieve a sufficiently useful level of analyols. 

The Relevance of Rule Induction 

With this general framework of constraints in mind, we will dis- 
cuss research that our group and others have conducted on a class of 
tasks that is presumed to ansess a psychological capacity for rule Induc- 
tion and Is commonly found on tests of aptitude and Intelligence, This 
set of Intercor related tasks Involves several task forms such as classifi- 
cation, scries extrapolation, analogy, and matrix tasks. These task 
forms simultaneously vary along content dimensions that include letters, 
numbers, words, and geometric figures as qhown in Figure 1. Spear- 
man (1923) considered such tasks as measures of^, and he viewed them 
as an index of the capacity to engage in intellectual processes that he re- 
ferred to as the "education*' of relations and correlates. Thurstone (1941) 
treated these taska as representative of a primary mental ability called 
Induction (I), and suggested that rule induction as a second-order factor 
might be identical with Spearman's In more r<»cent hierarchical ap- 
titude models, such tasks have been treated as measures of or fluid 
analytic ability (Brody & Brody, 1976), It seems clear that such rule 
induction tasks assess basic reasoning abilities that comprise a robust 
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uptltudo conitruci thAt h^n r«Uv«nt0 for n Urjiwr (Jomdin (if huiuAn p^r- 
(ormancf*. It h^t bo«»n Argued Oiai rul« Indiutlun |>r(K:i*m*i?fi Arn wliniUr 
to tho»e donunded In conc»pi fonnAtlon, aiuI thul tht?y ttn* roUted to tt niA 
Jor form of hunun problem iioMnK that rcnulti in the Acqiilnllion of hnuwl 
edge* 



SERItS COMPLETION |»RODLt.MS 

Number Sfft<< 

!l }} 15 .4 |Q 35 

43 <J0 '1 4- i5 "0 51 so 

ANALOGY PROBLEMS 
VtrbiJ 

Sugjr Sweet :: Lemon . 

Yellow Sour Ffjii Squecrc Tej 
Abaic ; Decline W'i\ 

Increjse Improve Blemish Poliih Wjne 

Geomei nc 



Our tentative view, then, of inductive reasoning tasks of the type 
found on mental ability tests ia that such tasks represent performance 
samples of the way in which an Individual makes use of existing knowledge 





Figure 1. Simple and complex forms of scries complelum and jna!o}(y tasks. 
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to iiulv^ circumncrlUfld pnthlmnfi wh(*rt> itoluiiitt) (l0pf»n(U tipon am AhAly«ih 

Ivrn dltirnt^ntu* WlUilrt intportAnt UrDitatlortii, rf<«rinttnv<i mt thuMt^ tAiikii 
hrtii cortniiitcfnUy curreUttxl with ACAilttruiv Achirvninrnl , and indivlduAt 
diffordncoN in tbti CAp4clty tu pnjjAUfi In nuv\\ Anrtlyann rtppt^Ar to Imvi* di- 
mcl ImpliCAtldnii for cornmordy rr>(|ulr<id clAHKriKirft It^Arrtirtu proc«itiittit. 

Wt> nHaU huw dc<Mcrlb(i a nuinbrr i)f roMn.irch findin^n on vArloun 
inductlvo roAMoninK tent taikn. Thin rmeArch in pristtonted in tho <orn\ 
of nuccinct done rlptiont And concluiiionii with Utttr of tho rxprr iinrntttl 
dtstail and CAvOAtu which arr in *ivAlUldc! pAprrn (nor rrfrrrncttn vitt^d 
bolow). 

Scricii Completion Problcmw 

Series completion itenii* am Nhown in Figvirr 1 arr found «)t Ntiv* 
cral devolupmontttl lovoU on nmity atandArdiArd .tptitudt* toNtn* Such 
Itcmi! inay be roprenented aa letter worim, number Hcrien, picture wrr- 
lea, or geometric figure aetioa problemn. In aU caaen, the taHk atruc- 
ture ia the aame: elcmenta corrtpriaing the scrica are ordered according 
to some apecific intoritem reUtionahip, and tho individuaTa taak ia to ex«* 
tract the baaic rolationahipa and gen^^rate, predict, or aoloct the next 
itcm(8) in the acrica. The acquiaition of aerial pattern concepta haa an 
extenaivc hiatory of paychological invcatigation. Of particular intereat 
for our purpoae here is the work of Simon and Kotovaky (1963; Kotovaky 
Ar Simon, J 973) on the analyaia of letter acrifa problcma of the type de- 
veloped by Thuratone and Thuratone for their Primary Mental Abilitiea 
teat battery. Simon and Kotovaky studied adult and adolescent perform- 
ance in thia type of taak, and they developed a computer simulation model 
to represent the cor^ponent proccsaca necessary for solution. 

One important aspect of the analysis of this task is the distinction 
between the declarative knowledge and the procedural knowledge or pro- 
cesses necessary for the task. The declarative knowledge base for such 
letter series problems is limited to knowledge of alphabetic order and to 
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r«UUiiniil cunc«ptii nuih 4^1 itli»iUily (nAiim l«iHi»r), tmxt (ih^ upM Uteor), 
And t>A(*KwArdH«n«Nt (or r«v««riit) iinlprinjt). Obviously, l«iH«r »niri«i»i |>roU« 
Urn* do HQt involvf? An t>xtDni4lv« di»cUrrtllv«s knnwlndnt* iompunpnt, 4nd 
it would not bo «»icpficlt»d \htki ininvl<lu4l difftiroitcrii would arin^ (ruu\ d0« 
cUrAtive knowl«dg« d«(ici«ncleii. 

Qlviin th«t \hp Ap|ir<)pri£tto dticUrAtivo knowl0d|;{«s iA 4V4lUbl«i, th« 
completion of Any Itfttnr uprlwn prutitrtn r(i()ulr«iii a not of bANh pr(ti;<«- 
duro« which at© hlerArthlcftUy urwAnUod. In th«» nlmuUtlun imidwl, thcrw 
Ari» two hAiiic routtn(?»{ a pAttorn ^tinrrAtor And a noqucncti Kont^rAtur. 
Thr' f'mt uf thi««« routine?*, iMtti^rn gtinvratton, Ktktx bo broken down into 
Ihrcio procQMtei: (a) d«ti?ctlcm of th<» lnl«rlett«r rrUtlonn for Iho given 
problom elfm*4i»itt, (b) une of Ihr rvlAliunAl tnformdtlon to cixtrAct Ihi* 
p«rlod length of ihti (»4ttr*rn within iht* problem, find (c) jjwnti rAtion of a 
pAttorn doicrlption or r\i\p involving both ihr retalionA and Ihr prriodiv 
utructurtt of U\P problem. Thi** rult* ttpacifylng thtf> pAtt.<«rn description 
• «rvoa Ai input to the ti0<iucnccr Konorator which applim it to thr current 
itAto of tha problem. And then rxtrApolAtos thc^ pattern to gc^norAtcr tlir 
Additional olemonts required for problem nolution. Differenceii in itrm 
difficulty And potential individual differrncRM in problem nolutiun can re- 
sult from the applicAtion of any or all of thcNe specific procvvneii. 

Concerning aources of task difficulty, a number of Hystrmatic 
properties of the individual items determine the difficulty of a problem. 
One Aapect related to the probability of error is the type of relation in- 
volved. Identity relations arc i^asier to detect than next reUtionn, witich, 
in turn, are easier than bacjcwardt»-next re latitinships* The difference in 
difficulty between extrapolating identity and next rolationshipH also varies 
as a function of the position of the relationship within a perio<l* 

These sources of error are readily explainable if one considers 
the requirements of working memory. Identity relationships do not place 
demands on working memory, whereas successive nonidentity relation- 
ships involve accumulating placekecpers. The longer the period length. 
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houd U)At wurKlhK nirinMry ItintU fii^y r0A^lkt><K For 0)<«(inpl<^, rhu 
l«lt«r Aiul number itrrl^i pr(4)l(;iu« in KtKur^ I (hff««r in liolh prrtinl t(in|]lh 
And |>Alt«irn coinpUitity « Thi? iivt^rAll )mttt«rn d^iic ription th4t i i>nMtiiutc*ii 
A |)rubl«iT) in thuH r^Utiiil to prohUin tli(fi<uhy. '\Up Unuth of ihv pAtti^rn 
clone riptiun, which in 4 functUin u( imriotl Uni^th, the typm uf r^iiitton* 
fihipii (nvolviid, itnti ihr rttitultin|i wurkinij n\Ptuury r^qui rf<rni»nt»i, Art* 
hiithly corrrlati*(l with problt^m errom. 'rhmti nrrum (An 4riM«^ from 
|Hirformanco inArlo()UAcic(ii ihAt differ AnumiJ IniUviduAlft in rrUtion dt^* 
Ipction, dlpicuv(<ry t»f ihp pr«ii»dic ittructurn, tontptrtinn of a p4ttf*rn dr* 
ncription, or in thr> rxtrApolAtictn provtinA, 

Connidrr now innt rtK tiuniit t^Kprrintrnt^tion hAnrii \i\ivt\ thin iiukI**!. 
It h«iii bec^n (liiiitoniit rAtctd tl\At Sinum And Koto\ nKy'i* rtuxlrt cd pi(*riAl < oin- 
plotiun ittimii pruvidoit a rt?Ai(t>ndbtr AVt ount id prrffirtnAni..r in thin tAnK, 
And thflir MirnuUtion of huii)An protcx oU providf^ix a (i^rtiAl v^Udation of 
th(? intKlrrl. Given n^tw a concrrn with thr crltrrittn of innt ructionAt trAct- 
Abilily, Anothrr wa'' CAn br conNidf^rrd in uhith HUch intxirlit cAn \w vaU- 
dated. Tho roanoninK r\inn an foUt wh* If thct proccmmi* cTrtbiKiird in a 
MinfiulAtion ntixlol ai«^ ^trniUr to thonc* uncd by hvtniAnn, then thuMO pro- 
con •rv n^y be trftinnbto fur individuAU whodo prrformanco rrprcnontn a 
low or intr rrnodiAtrr icvct of tank i onipotency • .Such trA(ninf{ nhoidd im- 
prove performAncc if the component pracc^?toa iiprcifitnl and taught are 
compAtlble with hum^n coK^^^^ve rttructuren. Mowrver, if thr procrnnrit 
are incompAtibte with auch ntructuren. then thr*rc will either be difficulty 
in training these proceHticii, or they will have no potiitivi* and perhapn a 
negative influence on performance. 

in an Attempt to provide Much an inntructioncil tcHt, a ntvidy w.itt 
con ; i4 trd tKat in\ ivnd direct and independent tr.iininj;; in discovery of 
relaii >hti ;)nd discovery of periodicity with a Nample of children fmrn 
grades :nc fhrotigh six (Holr.man, Glaaer. h l^rlleKrino, 1976). Both 
the training fjroup and a control group Aere then given a pn?- and post- 
test set of letter Hcries problems identical ir. rule structurci but viitial- 
izcd a*: different points in the alphabet. The results showed a signlticant 
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(llffl«uU fMllf^rnA, wh0r*»rtpi \Up runlrMl uroui* r*»rnrtini?«l ih«i 11411m mt rp- 
duv*»cl erram only «n tixp P^nivr Thu». ihi> irrtininu 4|»firo|iriAt*t|y 

fwnetUmf.<l wh«re (t wam moMK n*»ml^d.-thrti l«, when iniHvulu^li. f«nuHmt*trt»iJ 
more difficuU rc»Ultunii (»ml prublwmn, TUp «|UAllt<inv<« tUH^^rfnu bclwij^n 
i^ontrol And c>xp«rtin«inlAl cundiilunn ^Up *>u»Ktii*lfi Uml <^«|divU lrrtlriln,i i»n 
lh«» Identified componrnl proct^iiiK^ii nwy hrtvt? provided An InfuriUrtHHn 
mdnagemont utr^K^Ky th^l fAcllllatrd \uk\\pru de.*v rlpllon «ind eM rrt|iolttium. 

UitUted InntrucliunAl ntudirM that wmrr ci»ndiht*«fj un «t^rie» 4 lun- 
plrtlon perforniAnce nhowrd ah inf ?rrtiiittn betwrrti type t.f trrtminjj rtrul 
pwrfornwntn level. Sheer prrttlice on U»e teitl Henn* w«»i itiifru irni to pr«»- 
duce -IjinldtAfir ^ainN in perfonimnce aI lertAin lev^U •»( iniUi«l roinpr- 
trnce, but expllcll priM eB» irAUUnK more effpctlve than prA* tl* cr At 
lower levels of InllUI Ability. Thin iiource of interaction need* tii t)r 
more precisely determined by explicit AOalyriin of the prt pnn difrereoieii 
Among lndividu«.l» thAt defme different levrU of tA«k comjietence ^nd in- 
tellectviAl development. 

The trAinlnu atudy junt He»»crlbed reprt^nrotn a very n.tive t.»rni uf 
instructlonAl experln^entAtlon. No atte«npt waj* made lodeterrume rl.. 
pArtlcular componcntu of performance that were rcaponnibl** for thu <hf- 
fercnl levels of task con^petence. Tho procewa training Out Admin- 
Istered was the HAme for all individuaU, And no real effort was nmdc t<» 
match trAinint; to porformAnce nredn. Hut Uierr \» no farticuUr rcAson 
why future instructional «*tudie8 cannot uhi» miKlrU of iank |>e r(<»rmance 
AS A basis for both the initial diagnostic assesHmcrnt of individual cogni- 
tive fitrcngthjf and v/cakncssca and the aubjicqucnl iraininK deHigot-d lo 
improve cognitive skills. But so far, have at lea»l shown that a mo<l- 
<I of series completion performance can suK«rs»l component processes 



that contribute to item difficulty and that can be translated into a set of 
procedures for explicit process training. What remains to be shown is 
whether models of this kind provide useful frameworks for the analysis 
of individual differences in more generalizable inductive reasoning skill. 

We turn now to another task form presumed to assess the capabil- 
ity for Inductive Reasoning. 

Analogical Reai Ming Problems 

Of the riuxny tasks that are assumed to assess this capability, the 
analogy problem is the most pervasive. Analogy items as shown in Fig- - 
ure 1 have constituted a significant portion of intelligence tests over the 
entire course of the testing movement. Burt introduced the task in its 
familiar "A:B::C:D" format in a test published in 1911- Recently, Robert 
Sternberg (I977a, l977b) has provided a detailed review and discussion of 
the importance of analogical reasoning within the field of differential psy- 
chology, and the centrality of this type of reasoning with respect to the 
concept and measurement of intelligence can be found in the writings of 
individuals such as Spearman (1923) and Raven (1938). 

In the past few years. Spearman's theory has been expanded and 
refined in the more precise, experimentally founded theory presented by 
Sternberg. He has proposed and tested a theory of analogical reasoning 
that specifies several processes that are intended to apply across all an- 
alogical reasoning tasks. In Sternberg's analyses, emphasis is placed on 
developing general models of analogy solution and specifying individual 
differences in terms of latency pararpeters for the various processes in- 
volved. He obtained multiple correlations between . 68 and . 87 for esti- 
mates of process speed and gene-al reasoning scores from a stand*idized 
test battery. However, processing latency for the separate components 
was not uniformly related to general reasoning ability; in some cases in- 
dividuals with high general reasoning scores were slower on certain com- 
ponent processes. Also, the qualitative characteristics of these pro- 
cesses remain largely unspecified, and weT presumably have a poor 
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understanding of how the processes are executed, the information or con 
tent that must be processed, and how such information contributes to dif- 
ferences in item difficulty and performance errors. 

In our analysis of individual differences in analogy solution, we 
have attempted to determine the components of task performance that dif- 
ferentiate between levels of aptitude and how these components interact 
with differences in item structure or content. The task forms studied in- 
clude geometric and verbal analogies, which we will consider separately* 

Geometric analogy task . A starting point for understanding the 
declarative and procedural knowledge requirements in geometric analogy 
solution can be found in an artificial intelligence analysis of this task 
carried out by Evans (1968). He developed a computer program that 
solved a subset of the geometric analogies that appeared on the American 
Council for Education' examinations. 

The principal operations in Evans' program are the following: (a) 
decomposing the patterns that comprise the terms of the analogy into sub- 
patterns, (b) determining the transformations that relate the subpatterns 
in the A-B and C-D pairs of terms, and (c) selecting an answer. Decom- 
position of the A and B terms occurs first. This is accon^plished by com 
paring the figures and determining the common elements or subpatterns. 
Next, the program matches the subpatterns in the A and B terms and 
generates a set of transformation rules. The class of transformations 
recognized by the program includes: removing and adding constituents, 
rotating, reflecting, and spatial displacement of figures. Following the 
identification of the constituents of the terms and the transformations re- 
lating them, Evans' program decides among the a nswer alternatives by 
comparing the A-B transformation with each of the possible C-D options. 
The D term for which there is the greatest overlap in transformational 
rules is selected. 

The processes represented in Evans' program are compatible 
with Sternberg's general theory of analogy solution. In all cases, the 
individual, must encode tl^e analogy terms and infer, match, and test the 
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relationships among sets of terms. These processes are influenced by 
the amount and typs of information that must be processed to solve any 
given item. In geometric analogy items, our studies indicate that item 
difficulty is a function of both the number of elements and the number of 
transformations contained in an item, and that solution accuracy and la- 
tency depend on the total set of processes required to: (a) decompore 
complex figures into constituent elements, and (b) identify and order the 
transformations applied to each element (MulhoUand, Pellegrino, & 
Glaser, 1977). Identifying the transformations applied to individual ele- 
ments appears to require more processing time and results in more 
errors than identifying the elements themselves (elements usually being 
easily perceived geometric figures). Multiple transformations become 
particularly difficult when applied to a single element, for example, ro- 
tating, flipping, and changing the size of a single shape. Such items may 
be viewed as requiring an individual to operate on a series of partial pro- 
ducts, and maintaining and updating the intermediate products place ad- 
ditional demands on processing resources and memory capacity. With 
certain transformational combinations, the sequence in which the trans- 
formations are applied is particularly critical, and this may constitute a 
further difficulty since there is a need to maintain order information as 
well. 

The work just discussed; together with the geometric analogy ex- 
periments described by Sternberg {1977b) and the artificial intelligence 
analysis of Evans {I968), represent virtually all of the attempts at de- 
scribing performance in this task. It is obvious that this is only a begin- 
ning at understanding the sources of item difficulty and individual differ- 
ences in performance. The results are encouragingly clear because the 
items used in experiments are amenable to relatively precise-specifica- 
tions of the information represented in a problem and the rule to be in- 
ferred. However, test items do not always have one unique rule or repre- 
sentation. And the solution of such "ambiguous" items depends heavily 
on the cognitive representation that an individual provides for the ele- 
ments of an item. For example, in the second geometric analogy item 
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in Figure 1, the transformation between the triangles of the A-B terms 
was accomplished (represented) by individuals in different ways; (a) by 
moving the small "X" down to the empty space in the triangle, or (b) by 
rotating the entire large triangle 120° clockwise and then flipping it over 
on a vertical axis* Each transformation when applied to the C term dic- 
tates the choice of a different answer that is given in the option set. The 
completeness of the representation, the method employed to achieve the 
representation and select an answer, and the role of such factors in in- 
dividual differences remain large question marks. 

Further, it must be noted that performance in this task probably 
involves a considerable amount of procedural knowledge involving strate- 
gic flexibility and expertise. - The strategic flexibility may be particular- 
ly important when it comes to handling complex items. Some items place 
severe demands on the processing resources and working memory capac- 
ity of an individual, and 'expertise may include the ability to shift process- 
ing strategies in order to circumvent some of the working memory prob- 
lems that arise when using a strategy that is optimal for less difficult 
items* (For example, an exhaustive initial inference process applied to 
the A-B pair, in which all the element transformation relations are ini- 
tially stored in working memory, may he less efficient than a process in 
which individual element transformation combinations are assessed one 
by one across the entire problem. ) Whether such strategy shifts occur 
and their relationships to task proficiency remain speculations. 

Verbal analogy task . For geometric and alphanumeric series or 
analogy problems, the declarative knowledge base neceesary for solution 
is relatively finite and easy to identify. Ambiguity may be associated 
with possibilities of more than one transformational representation, but 
each is a definite representation that yields a definite solution. In con- 
trast, verbal analogy problems have tremendous variability in the indi- 
vidual elements that may be encountered and in the representation of ele- 
ments and their relationships, for example, the interpretation of the word 
"wax" in the second verbal analogy in Figure 1. Furthermore, identifying 
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particular rule relating two terms does not mean that one can always 
apply that rule to generate an exact answer. These aspects of verbal 
analogy problems make it more difficult to specify the processing com- 
ponents and task factors that contribute to differences in item solution 
and difficulty. 

In attempting to analyze the components of performance, it is ini- 
tially useful to talk about the base or stem of the item separately from 
the set of alternative completion terms. Analyses of verbal analogy 
items reveal that the majority of verbal analogies can be classified as 
representing a limited set of basic types of semantic relations (e.g. , 
Ingram & Pellegrino, 1977; Whitely, 1976). Items containing certain 
types of relationships such as location or function are verified as true or 
false faster and have fewer errors than Icems involving more abstract re- 
lationships such as class membership. 

However, this classification process does not immediately lend it- 
self to strong predictions about the difficulty of solving an item, since it 
does not specify the total set of features that must be processed to define 
the rule for a given item. The semantic characteristics of the item as 
represented in the stem constitute only part of the semantic task factors 
contributing to performance. For any item, there also is a set of possi- 
ble answer alternatives that vary in their semantic appropriateness with 

c 

respect to matching the semantic features inferred from the stem of the 
item. These differences between potentially acceptable responses may 
be considered as representing a "goodness of fit" or semantic distance 
factor referred to in studies of semantic memory (e. g, , Rumelhart & 
Abrahamson, 1973; Smith, Rips, 8t Shoben, 1974). 

The semantic appropriateness of alternative answers affect per- 
formance through the likelihood of their acceptance, and the difference in 
features associated with a set of alternatives affect the accuracy and speed 
of any choice among them. The experimental data that we have collected 
thus far on the effect of these item features on performance reveal that 
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there is rapid and accurate rejection of semantically inappropriate com- 
pletion terms, and a slower and l^ss likely acceptance of semantically 
less appropriate completion terms. If errors are made, they seem to 
be localized in a later stage of processing where additional processing 
of ambiguous or complex items occurs. 

As a general conclusion, performance in verbal analogy tests is 
consistent with general models of semantic processing. The acceptance 
or rejection of any given alternative is a function of the congruence be- 
tween the semantic features that define the A*B and C-D rules. While 
this is the case for all items, the time and likelihood of accepting the 
'*best" alternative for an item varies considerably across items. And 
this variability is partially accounted for by the type of relationship in- 
volved and the degree of constraint on the set of possible answers for an 
item (Pellegrino & Ingram, 1977). 

The question that now must be considered is whether these char- 
acteristics of performance on test tasks differ as a function of aptitude 
level. Over all items, the high aptitude individuals (as defined by test 
scores) respond faster than the low group. However, high aptitude indi- 
viduals tend to spend more time in processing the initial terms of an item. 
If this response time Is considered relative to the total time spent in pro- 
cessing an item, then a pattern emerges in which skilled individuals s^end 
proportionately more time in initial encoding and inference processes' and 
less time in subsequent decision and response processes (see also Stern- 
berg, 1977b). 

• If it is assumed that the initial encoding and inference processes 
involve the specification-of a rule based upon a set of features relating 
the A, B, and C terms of an iter. ., then the specificity of that rule mu£;t 
be different in the case of high aptitude and less-skilled individuals. Two 
findings confirm this notion. First, the amount of time required to speci- 
fy a precise rule for an item should vary as a function of the number of 
features or complexity of the rule. This pattern of longer latencies for 
more complex and difficult items was observed'for the high aptitude 
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individuals, with inconsistent latency differences shown by the less-skilled 
individuals. Second, a more precisely specified rule should allow one to 
reject false alternatives rather rapidly, and such a result occurred only 
in the case of high scoring individuals* 

In general, the data suggest that individuals with high aptitude test 
scores specify more precisely the set of semantic features representing 
the interrelationships among the individual components of the item, and 
that this difference in the quality of encoding gives rise to different latency 
and error patterns. Protocol analyses of individual subjects to further in- 
vestigate this issue have highlighted a number of performance characteris- 
tics that differentiated between items that individuals find to be of high and 
low difficulty (Heller & Pellegrino, 1978). Items that are relatively easy 
lend themselves to a solution process in which the relationship is readily 
specifiable, and a potential completion term for the item is easily gener- 
ated. Thus, the process of solution follows a generate and test model in 
which the processing of the alternatives involves a simple search for the 
hypothesized answer. This can be called a "working-forward" strategy 
and is illustrated in the protocol shown in Table 1 for the item TearCoffee:: 

Bread: . in contrast, difficult items for individuals are ones in 

which the relationship is not well specified, and there is difficulty in gen- 
erating a potential answer. In this case, solution is guicied by the set of 
alternatives and the relationship is defined by working backward from 
these alternatives. The protocol of such a solution process is shown in 

Table 2 for the item Subject:Citizen::King: , Thus, items 

appear to vary on a continuum from a hypothesis generation and test pro- 
cess for easy items where there is little change in the level of feature 
analysis, through to a process for difficult items that is largely driven 
by the alternative set with constant redefinition of the possible relevant 
features of the problem. Thus, depending upon the difficulty of an item 
for an individual, verbal analogy solution varies from a simple, sequential 
execution of component processes to complex combinations of working for- 
ward and working backward strategies with increasing reliance on the op- 
tion set. These latter cases either involve ambiguous relationships or the 
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presence of two or more options at similar degrees of analogical appro- 
priateness, and conceptualization of the relationship often is constructed 
through a gradual, iterative process. 



Table ! 

Solution Protocol for "Simple" Analogy 



Ajialogy aemenis Presented Solver's Response 



TEA: COFFEE;: BREAD: 




Tea is to coffee as bread is to . . . rolls because 
tea and coffee, they're both drinks, and they're about 
the same thing, just two different names for two 
different dnnks. and a bread and a roll would be 
about the same-two different rames for the same 
thing. 


MILK 


(Reject) 


That doesn't fit. it's a dnnk. 


BUTTER 


(Reject) 


Butter is something you put on bread, that doesn't fit. 


ROLLS 


(Accept) 


That's good. 


JAM 


(Reject) 


It's like butter, something you put on bread. It 
wouldn't fit because >ou don't put coffee on tea 
or in tea. 
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Table 2 

Solution Protocol tor Xomplcx** Analogy 



AnaJogy Hcmcnis Prcscnicd 



Solver's Resoonse 



SUBJECT: CITIZEN:: KING: 



RULE 

KINGDOM 

PRESIDENT 

KNIGHT 
RULE 

PRESIDENT 



Subject . . . king ... If you're subject to something 
... no. that wouldn't have anything to do with ii. 
I don't know about this one. what could fit in there. 

Researcher: Do you see any relationships between anything? 

Well, the king would sort of subject the people to 
stuff $0 maybe it would be something like the people 
who are under him, 

(Accept) Citizens are subjects and a king rules-that couJd be 

true. I sort of think of it in a way like the citizens 
are people thai are subj-ct to the rule and the king 
is the one that does the ruling, 

(Reject) A king would rule over the kingdom-bui the citizens 

are subject-oooul I don't know, but I think citizens 
are subjected to the ruler. 

(Accept) Tlie king and the president would be the ones that 

rule. Subjects and citizens would be tfie ones that 
they rule, the people that they rule. 

^^^j^'^^) I don't see what knight would have to do with it. 

^^^i*^^^^ I'd say that's raise because rule is what the king 

would do. But citizen and subject, it doesn't say what 
they do. they're sort of like the same thing.' 

*^'^'-^^P^' A king's like a president, a subject's like ■ citizen. 



ImplicationB 



Individual differences and item difficulty . At the beginning of 
these remarks, certain criteria were stated that are useful for evaluating 
the results of an analysis of tasks representing a major aptitude construct. 
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One criterion is whether the results of the analysis move us closer to an 
analytic scheme that can be used to diagnose performance deficiencies* 
and a second is whether the sources ox individual differences suggest 
testable instructional hypotheses. A& is obvious* this paper has been 
primarily devoted to theoretical and empirical analyses of the task and 
item properties that influence performance on aptitude test tasks, and 
we are o iy beginning to understand the sources of individual differences. 
Thus, at this point, we can only speculate about the instructional impli- 
cations of individual differences in inductive reasoning skills. However, 
the work so far clearly provides a basis for individual differences analy- 
ses, partic^arly with respect to the sources of task difficulty. There 
appear to be two major factors that contribute to task difficulty; thesi; in- 
clude: (a) the complexity of the rule to be inferred, and (b) the variability 
or initial ambiguity in the possible rules that may be inferred. 

In all of the tasks that employ nonverbal stimuli, rule complexity 
is reflected directly by the number of different operators that must be 
represented in working memory. The greater the number of transforma- 
tions that comprise a rule to be inferred and applied, the larger the in- 
crease in both solution latency and error probability. This is due to dif- 
ficulties in assembling and maintaining a complete description of the 
element-operator combinations. Such descriptions may exceed memory 
capacity, and only partial representations of the rule to be induced and 
applied may be established. 

In verbal items, rule complexity is more difficult to measure 
since it is not directly manifested by any overt problem features. Also, 
verbal items seem to place somewhat different demands on working mem- 
ory, particularly with respect to maintaining information about past and 
current hypotheses about the relevant semantic features that comprise an 
item's rule. The changing nature of the semantic feature set that occurs 
as new item terms are encountered is related to what can be called repre - 
sentational variability . In verbal analogies, item difficulty is heavily in- 
fluenced by variability resulting from the conceptual richness and abstract- 
ness of the individual terms and relations. Verbal concepts activate 
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extcniivc ■emantic feature leti, and varying subsets of these features 
may be included as part of the problem representation. The ability to 
enterUin different problem representations and modify them during the 
course of solution represents a significant aspect of performance. This 
ii also true for nonverbal items that often have two or more possible 
represenUtlons for the rule governing the problem. 

The factors of rule complexity and represenUtional variability 
provide a scheme within which to consider sources of individual differ-, 
cnces. The limited individual difference daU that we have discussed fo- 
cus primarily on the representational variability factor as manifested in 
verbal analogies. The data indicate that skill differences in an under- 
graduate population are associated with: (a) processes of establishing a 
reasonably weU-defined problem representation, (b) the subsequent uti- 
lization of that representation as a basis for selecting among alternatives, 
and (c) modifying the reprcsenUtion as necessary. The time spent es- 
Ublishing an initial representotion (or representations) may differ as a 
function of aptitude level, but Utency differences may be less imporUnt 
than the particuUr repre8enUtion(s) achieved. Indeed, there is evidence 
in our daU and others* that high aptitude individuals, who presumably have 
more elaborate semantic memory structures, may encode more item fea- 
tures and uke more time in this aspect of processing, but with subsequent 
faciliUtion in the speed and accuracy of selecting among alternatives. 

Instruction. The different Usk analyses that we and others have 
carried out suggest three potential areas for instructional research. But, 
before making these suggestions, we must remind ourselves about what 
the goal of our enterprise Is. It is certainly not to produce merely high 
scores on aptitude tests. (Thin can be attempted by such means as buy- 
ing a practice booklet for the MiUer Analogies Test that provides drill on 
the cUsses of relations that appear on that test. ) Rather, the eventual 
goal is to increase individuals* proficiency in inductive reasoning in the 
context of subject matter knowledge in order to improve learning and 
attained competence. Essentially, we are interested in a fourth R that 
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should pervade the other three--Reading, Writing, (A)rithmetic, and 
Reasoning. 

With this in mind, the firtt general instructional research sugges- 
tion points to the fact that there is a declarative, content knowledge base 
that must be available to perform any reasoning task. Declarative knowl- 
edge for rule induction tasks involves the basic elements and possible 
transformations for the particular content area. This is a large, rela- 
tively unbounded set of knowledge in the case of verbal reasoning, and 
thus, it poseii a nontrivial instructional problem. Nevertheless, it might 
be possible to teach the necessary declarative knowledge for a restricted 
content domain, and then use this to permit the exercise of inductive rea- 
soning processes that involve a search for relations among relevant ele- 
ments of knowledge that influence further learning. 

The second target for instruction would involve factors associated 
with the rule complexity dimension. This might involve two differen^ 
types of instruction, one being instruction in procesoes associated with 
storing, retrieving, and manipulating information in working memory. 
Whether sucn instruction is feasible is unclear, but work is proceeding 
on understanding these memorial processes. The second potential fovm 
of instruction would involve executive strategies for organizing, control- 
ling, and monitoring the analysis of problem features during the course 
of solution. Such procedures may substantially reduce memory load prob- 
lems, and they can be viewed as forms of procedural knowledge. Such 
procedural knowledge may be instructable, and minimal evidence for this 
is that partial procedures such as how to discover periodicity in series 
completion problems can be readily taught to children, and such knowl- 
edge affects performance within the task. 

The third target ^or instruction involves factors associated with 
the representational variability dimension. In this case, the form of in- 
struction may be linked to knowledge about general aspects of problem 
solving such as defining the relevant problem space for a task and then 
using information within the problem space to help restructure the problem. 
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The extent to which and how luch problem-solving skills can be opera- 
tionalized and taught is also unclear at present. But our guess is that 
knowledge of instructional procedures for developing these skills c-.n be 
obtained from study of the carefully sequenced problem sets that an ex- 
pert tutor presents to students. 

Finally, it can be concluded that systematic task analytic efforts 
of the cognitive processes involved in sets of tasks such as we have de- 
scribed should help make it possible to understand the link between rule 
induction skill and instructional situations that influence this skill. In 
this way, a contribution might be made toward generally improving the 
success in learning that aptitude tests attempt to predict. Consider the 
possible relevance and generality of processes that have been identified 
in the small set of studies of inductive reasoning tasks. The rule induc- 
tion processes involved in analogical reasoning and series completion 
appear to be similar to many forms of problem solving and concept for- 
mation. The essence of this similarity is the ability to search for rela- 
tions among elements resulting In new interconnections between concepts 
stored in memory. Consistent with this contention, it has been argued 
that one of the learner's essential roles in classroom learning is to recog- 
nize the Aructural form or pattern of the facts conveyed by instruction 
and to detect relations between this newly communicated material and the 
material already existing in a semantic network in memory (Norman, 
Centner, &c Stevens, 1976). 

Whe n we are able to specify intellectual abilities in terms of psy- 
chological processes, then we have information that enables us to do 
more than predict performance on a criterion task. We then have infor- 
mation that provides a basis for doing something about performance either 
by engaging in specific process training designed to improve performance 
or by changing the learning situation to make the attainment of criterion 
performance more likely. An increasing number of studies are being 
carried out to determine the direct instructability of specific processes 
that underlie intelligence and aptitude. Much of this work is being done 
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in the context of reaearch on mental retardation (Belmont & Butterfield, 
1977; Brown, 1974; Campione & Browo, in press); However, testing the 
limits of such training on a wider spectrum of abilities has not yet begun. 
Experimental studies investigating the possibility of changing the learning 
situation to adapt to individual differences have been largely represented 
by studies of aptitude-treatment interaction. But aptitudes and treatments 
have been rarely analyzed in terms of the relationships of similar under- 
lying performance processes, and this probably accounts for the many 
negative findings in this area (e. g. , Cronbach & Snow, 1977). Success 
might be expected when we have a more refined process analysis that re- 
lates individual capabilities and learning requirements. 

The potential benefits that can be derived from an understanding 
of the cognitive components of individual differences are consistent with 
the nature and purposes of education. It is no longer possible to consider 
testing only as a means of determining which individuals are already 
adapted to or who have the potential for adapting to mainstream educa- 
tional practice. Society's current goal is to reverse this sequence of 
adaptation; rather than requiring individuals to adapt to means of instruc- 
tion, the desired objective is to adapt the conditions of instruction to in- 
dividuals to majcimize their potential for success (Glaser, 1977). 

This objective can be realized if learning can be designed to take 
account of an individual's profile of cognitive skills. If we analyze the 
performance requirements of various school activities and then analyze 
the skills that individuals bring to these task environments, we should be 
able to match the two. Learning could then be assisted in two ways. 
First, the cognitive skills that individuals bring to schooling could be 
matched to various instructional environments that utilize these skills. 
Second, the cognitive skills of an individual could be improved to meet 
the demands of the instructional environments that are available. Pro- 
viding for both the development of cognitive skills and accommodating to 
different cognitive capabilities offers maximum adaptability for enhanc- 
ing the likelihood of effective education. 
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